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■  Case-based  reasoning  (CBR)  is  a  problem¬ 
solving  process  in  which  a  new  problem  is 
solved  by  retrieving  a  similar  situation  and 
reusing  its  solution.  Transfer  learning  occurs 
when ,  after  gaining  experience  from  learning 
how  to  solve  source  problems ,  the  same  learner 
exploits  this  experience  to  improve  performance 
and  learning  on  target  problems.  In  transfer 
learning,  the  differences  between  the  source  and 
target  problems  characterize  the  transfer  dis¬ 
tance.  CBR  can  support  transfer  learning  meth¬ 
ods  in  multiple  ways.  We  illustrate  how  CBR 
and  transfer  learning  interact  and  characterize 
three  approaches  for  using  CBR  in  transfer 
learning:  (1)  as  a  transfer  learning  method,  (2) 
for  problem  learning,  and  (3)  to  transfer  knowl¬ 
edge  between  sets  of  problems.  We  describe 
examples  of  these  approaches  from  our  own  and 
related  work  and  discuss  applicable  transfer  dis¬ 
tances  for  each.  We  close  with  conclusions  and 
directions  for  future  research  applying  CBR  to 
transfer  learning. 


Observations  of  human  reasoning  motivate  AI  research  on 
transfer  learning  (TL)  and  case-based  reasoning  (CBR). 
Our  ability  to  transfer  knowledge  and  expertise  from 
understood  domains  to  novel  ones  has  been  thoroughly  docu¬ 
mented  in  psychology  and  education  (for  example,  Thorndike 
and  Woodworth  1901;  Perkins  and  Salomon  1994;  Bransford, 
Brown,  and  Cocking  2000),  among  other  disciplines.  Transfer 
learning  uses  knowledge  learned  from  solving  tasks  from  a 
source  domain  to  enhance  an  agent's  ability  to  learn  to  solve 
tasks  from  a  target  domain.  The  differences  between  the  source 
and  target  problems  characterize  the  transfer  distance.  Case- 
based  reasoning  transfers  problem-solving  knowledge  from  spe¬ 
cific  examples  or  episodes,  called  cases,  to  new  problems. 

While  researchers  typically  work  within  each  of  these  fields 
independently,  the  purpose  of  this  article  is  to  summarize  how 
case-based  reasoning  can  be  applied  to  transfer  learning.1  Our 
analysis  reveals  three  approaches  for  applying  CBR  to  transfer 
learning:  (1)  CBR  as  a  transfer  learning  method,  (2)  CBR  for 
problem  solving,  and  (3)  CBR  to  transfer  knowledge  between 
the  domains.  These  correspond  to  using  CBR  for  solving  entire 
transfer  tasks  or  acting  as  a  component  within  a  transfer  learn¬ 
ing  system.  The  transfer  distance  provides  a  new  metric  for  CBR 
researchers  to  assess  the  robustness  of  their  systems.  Further¬ 
more,  each  CBR  approach  has  implications  for  the  importance 
and  interpretations  of  different  transfer  learning  metrics. 

We  begin  with  an  overview  of  transfer  learning,  case-based 
reasoning,  and  the  three  approaches  for  applying  CBR  to  trans¬ 
fer  learning.  We  provide  examples  of  each  from  our  research  on 
physics  problem  solving  and  controlling  a  player  in  a  football 
simulation  and  also  describe  related  work.  We  close  with  a  dis¬ 
cussion  concerning  the  applicability  of  each  approach  over  dif¬ 
ferent  transfer  distances,  how  transfer  learning  assists  CBR 
research,  and  some  directions  for  future  research. 
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Combining  Two  Methods 
for  AI  Research 

In  this  section,  we  first  describe  the  transfer  learn¬ 
ing  and  case-based  reasoning  frameworks.  Using 
these  frameworks,  we  define  the  three  approaches 
for  applying  CBR  in  transfer  learning  systems  at 
the  end  of  this  section. 

Transfer  Learning 

Machine  learning  has  traditionally  focused  on 
learning  from  a  blank  slate  in  isolated  tasks.  This 
differs  substantially  with  the  way  humans  learn; 
people  frequently  leverage  experience  gained  from 
learning  one  task  to  improve  their  performance  on 
different,  novel  tasks.  Transfer  learning  is  the 
process  of  recognizing  and  applying  knowledge 
and  skills  learned  from  one  or  more  previous 
(source)  problems  to  more  efficiently  or  effectively 
learn  to  solve  novel  (target)  problems.  Methods  for 
transfer  learning  hold  the  promise  of  being  exceed¬ 
ingly  useful;  they  could  dramatically  decrease  the 
amount  of  training  required  by  successfully 
employing  knowledge  obtained  from  different,  but 
related,  problems.  This  promise  motivates  the 
development  of  computational  models  for  transfer 
learning,  which  has  been  the  focus  of  workshops  at 
NIPS-05,  ICML-06,  AAAI-08,  and  NIPS-09,  in  addi¬ 
tion  to  a  large  amount  of  work  summarized  in 
recent  surveys  (Taylor  and  Stone  2009;  Torrey  and 
Shavlik  2009;  Pan  and  Yang  2010). 

Figure  1  summarizes  a  conceptual  model  of 


transfer  learning  and  its  empirical  evaluation, 
where  the  domain  representations,  tasks,  perform¬ 
ance  metrics,  and  environments  may  all  differ 
between  the  source  and  target  problems.  Transfer 
learning  involves  three  steps:  (1)  learning  in  the 
source,  (2)  transferring  the  learned  knowledge  from 
source  to  the  target,  and  (3)  learning  in  the  target. 

A  transfer  learning  evaluation  compares  an 
agent's  performance  on  a  task  defined  in  a  target 
domain  after  learning  on  a  task  or  tasks  from  the 
source  (the  transfer  condition)  to  the  agent's  per¬ 
formance  without  any  source  experience  (the  non¬ 
transfer  condition).  Common  performance  meas¬ 
ures  include  initial  advantage,  learning  rate,  and 
asymptotic  advantage.  Initial  advantage  (or  jump 
start)  is  the  initial  increase  in  an  agent's  perform¬ 
ance  resulting  from  transfer.  Learning  rate  is  a 
decrease  in  the  time  required  to  reach  a  particular 
performance  level,  particularly  asymptotic  per¬ 
formance.  This  is  usually  measured  using  /c-step 
regret  (Kaebling,  Littman,  and  Moore  1996). 
Asymptotic  advantage  is  that  the  agent's  final  per¬ 
formance  may  be  improved  through  transfer. 

Differences  between  the  source  and  target  prob¬ 
lems  may  be  categorized  by  their  transfer  distance. 
Defining  common  transfer  distances  across  a  wide 
range  of  tasks,  for  example,  physics  problem  solv¬ 
ing  and  strategy  games,  is  quite  challenging.  Typi¬ 
cally,  individual  researchers  define  transfer  dis¬ 
tance  for  particular  evaluations.  For  example,  Pan 
and  Yang  (2010)  proposed  a  new  categorization 
based  on  two  assumptions  made  in  traditional 
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Figure  2.  The  Case-Based  Reasoning  Cycle. 
Adapted  from  Lopez  de  Mantaras  et  al.  (2005). 


machine-learning  studies.  In  particular,  traditional 
machine  learning  assumes  that  the  task  to  be  per¬ 
formed  (that  is,  classes  and  class  objective  func¬ 
tions)  and  the  domain  of  performance  (that  is,  fea¬ 
ture  spaces  and  instance  distributions)  do  not 
change  among  the  source  and  target  problems.  In 
transfer  learning,  these  assumptions  are  relaxed; 
the  source  and  target  problems  may  involve  differ¬ 
ent  tasks  and  domains.  Refining  definitions  of 
transfer  distance  is  essential  to  maturing  the  field 
of  transfer  learning,  and  we  refer  to  this  issue 
repeatedly  in  this  article. 

Case-Based  Reasoning 

Case-based  reasoning  is  a  problem-solving  process 
in  which  inferences  about  a  situation  are  drawn 
from  individual  instances  called  cases.  While  the 
roots  of  CBR  lie  in  observations  of  human  reason¬ 
ing  (Schank  1982;  Kolodner  1993),  this  discipline 
is  now  aligned  closely  with  computer  science.  CBR 
research  today  focuses  on  the  study  of  algorithms, 
representations,  and  their  applications  for  a  large 
variety  of  analysis  and  synthesis  tasks  (Aamodt  and 
Plaza  1994;  Lopez  de  Mantaras  et  al.  2005).  For 
example,  there  have  been  significant  CBR  contri¬ 
butions  on  recommender  systems,  legal  reasoning, 
textual  reasoning,  and  planning  tasks  (Aha,  Mar¬ 
ling,  and  Watson  2005).  Current  research  venues 
include  a  dedicated  conference  (ICCBR),  several 
annual  CBR-focused  workshops,  and  other  Al  and 
cognitive  science  venues.2 

The  traditional  case-based  problem-solving 
cycle,  shown  in  figure  2,  includes  four  steps.  Given 


a  new  problem  p  to  solve,  this  process  begins  by 
retrieving  one  or  more  similar  cases  (that  is,  prob¬ 
lem-solution  pairs)  from  its  memory,  or  case  base, 
and  reusing  their  solutions.  Solution  reuse  may 
require  an  adaptation  of  the  retrieved  cases'  solu¬ 
tions  for  use  in  solving  p.  The  proposed  solution  s 
may  be  revised  in  light  of  its  performance.  Finally, 
the  pair  (p,  s)  may  be  retained  as  a  new  case.  In  this 
manner,  CBR  systems  learn  by  accumulating  cases 
and  refining  models  that  support  their  four  steps. 

Theories  of  analogy  also  computationally 
explore  reasoning  from  cases.  The  structure-map- 
ping  theory  (Gentner  1983)  defines  analogy  as  an 
alignment  process  between  two  structured  repre¬ 
sentations  resulting  in  inferences  about  the  target 
from  the  source.  This  alignment  process  may  be 
used  for  retrieval  and  reuse  within  a  CBR  system. 
We  introduce  analogy,  and  its  connection  to  CBR, 
here  because  it  plays  an  important  role  in  a  num¬ 
ber  of  the  transfer  learning  methods  discussed  in 
this  article,  either  within  CBR  systems  or  as  basis 
for  potential  mappings  between  domains. 

Approaches  to  Transfer  Learning 
Using  Case-Based  Reasoning 

Recall  transfer  learning  involves  three  steps:  learn¬ 
ing  in  the  source,  transferring  the  learned  knowl¬ 
edge  from  source  to  the  target,  and  learning  in  the 
target.  We  categorize  CBR  approaches  for  transfer 
learning  by  which  transfer  learning  steps  are  per¬ 
formed  by  the  CBR  cycle.  We  identify  three  types 
of  approaches,  shown  in  table  1,  for  applying  CBR 
in  transfer  learning  systems:  (1)  as  a  transfer  learn- 
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Source 

Learning 

Knowledge 

Transfer 

Target 

Learning 

Examples 

CBR  as  a  transfer 
learning  method 

V 

Hinrichs  and  Forbus  (2007),  Sharma  et  al. 

(2007),  Klenk  and  Forbus  (2009) 

CBR  for  problem 
learning 

S 

X 

Wu  and  Dietterich  (2004),  Shi  et  al.  (2009), 

Aha  et  al.  (2009) 

CBR  to  transfer 
knowledge 

X 

X 

Fui  and  Stone  (2006),  Kuhlmann  and  Stone 
(2007),  Shi  et  al.  (2009),  Konik  et  al.  (2009), 
Hinrichs  and  Forbus  (2011) 

Table  1.  Applications  ofCBR  in  Transfer  Learning  Systems. 


ing  method,  (2)  for  problem  learning,  and  (3)  to 
transfer  knowledge.  After  defining  each  approach, 
we  present  examples  of  them  from  our  research 
and  related  work. 

CBR  as  a  Transfer  Learning  Method.  CBR  can  be 
used  directly  as  a  transfer  learning  method.  Recall 
that  all  CBR  methods  involve  transferring  knowl¬ 
edge  from  prior  cases  to  new  problems.  In  CBR  as 
a  transfer  learning  method,  the  CBR  cycle  accounts 
for  all  three  steps  of  transfer  learning.  For  example, 
the  learned  and  transferred  knowledge  could  be 
the  case  base  after  training  on  source  problems. 
During  learning  on  target  problems,  the  same  CBR 
cycle  can  be  used  to  solve  problems  in  the  target, 
updating  the  same  case  base.  Thus,  the  CBR  system 
is  unaware  that  it  is  being  evaluated  for  transfer 
learning  and  makes  no  distinction  between  source 
and  target  cases.  For  these  systems,  transfer  dis¬ 
tance  and  initial  advantage  provide  a  useful  metric 
for  evaluating  the  retrieval  and  reuse  mechanisms 
of  the  CBR  system. 

CBR  for  Problem  Learning.  CBR  can  be  used  for 
problem  learning  (source,  target,  or  both).  In  this 
approach,  the  source  and  target  cases  are  separated 
and  treated  distinctly  by  the  system.  To  be  a  trans¬ 
fer  learning  system,  the  CBR  system  must  be  inte¬ 
grated  with  another  component  to  perform  knowl¬ 
edge  transfer  between  the  source  and  target 
problems.  Transfer  learning  provides  structure 
guiding  the  construction  and  evaluation  of  inte¬ 
grated  CBR  systems. 

CBR  to  Transfer  Knowledge.  CBR  methods  can  be 
used  for  transferring  knowledge  from  source  to  tar¬ 
get.  These  approaches  either  use  a  full  CBR  cycle  to 
modify  source  instances  for  use  in  target  learning 
by  another  algorithm,  or  they  create  analogical 
mappings  between  problem  domains  to  support 
the  transfer  of  abstract  knowledge.  These  methods 
must  be  integrated  with  a  learning  mechanism  to 
perform  transfer  learning.3  Thus,  these  approaches 
convert  established  learning  methods,  such  as, 


hierarchical  skill  learning  (Nejati,  Langley,  and 
Konik  2006)  or  Q-learning  (Watkins  1989),  into 
transfer  learning  methods.  These  methods  support 
transfer  distances  in  which  the  problem  and  solu¬ 
tion  representations  for  the  source  and  target 
include  different  relations,  types,  and  quantities. 

In  the  next  three  sections,  we  discuss  each 
method  in  more  detail  along  with  an  example 
from  our  research. 

CBR  as  a  Transfer  Learning 
Method:  AP  Physics 

To  illustrate  how  a  CBR  method  may  be  directly 
used  as  a  transfer  learning  method,  we  describe 
Klenk  and  Forbus's  approach  for  advanced-place¬ 
ment  (AP)  physics  problem  solving  (2009a). 
Physics  problem  solving  requires  reasoning  over  a 
wide  range  of  entities  and  scenarios.  While  the 
authors  sidestep  natural  language  understanding 
by  using  predicate-calculus  representations,  the 
translation  process  leaves  the  everyday  concepts  in 
place.  That  is,  balls,  buildings,  astronauts,  boxes, 
baseball  bats,  flying,  falling,  and  pulling  all  appear 
in  the  formal  problem  descriptions.  They  used  a 
subset  of  the  ResearchCyc  (research.cyc.com) 
ontology  containing  over  30,000  concepts.  Under¬ 
standing  the  relevant  abstractions  and  assump¬ 
tions  for  a  physics  problem  stated  as  an  everyday 
situation  is  a  difficult  problem,  in  part  because 
modeling  decisions  are  contextual.  For  example,  a 
coin  falling  off  a  building  can  be  considered  to  be 
a  point  mass.  However,  when  modeling  the  exact 
same  coin  spinning  on  a  table,  it  cannot  be  con¬ 
sidered  a  point  mass  since  its  shape  and  size  must 
be  considered. 

Solving  Physics  Problems  Using 
Analogical  Model  Formulation 

Given  a  scenario  from  a  physics  problem,  an  intel¬ 
ligent  problem  solver  can  use  model  formulation 
(Falkenhainer  and  Forbus  1991)  to  construct  a  sce- 
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nario  model,  which  consists  of  the  relevant 
abstractions,  assumptions,  and  equations  neces¬ 
sary  for  answering  the  question.  An  important 
contribution  of  the  qualitative  reasoning  commu¬ 
nity  has  been  formalizing  this  process  (Rickel  and 
Porter  1994).  While  successful  in  engineering 
applications,  these  approaches  are  limited  in  that 
they  focus  on  abstract  scenarios,  require  complete 
and  correct  domain  theories,  and  ignore  learning. 
To  overcome  these  limitations,  analogical  model 
formulation,  or  AMF  (Klenk  and  Forbus  2009a), 
builds  scenario  models  of  everyday  situations 
based  on  experiences.  Analogical  model  formula¬ 
tion  incrementally  learns  by  accumulating  exam¬ 
ples  and  making  effective  use  of  them,  even  when 
its  knowledge  is  incomplete. 

Analogical  model  formulation  was  evaluated  as 
a  transfer  learning  method  for  advanced-place¬ 
ment  physics  problem  solving  using  460  AP 
physics-style  problems  created  by  the  Educational 
Testing  Service  (ETS)  and  Cycorp.  These  problems 
were  generated  using  variations  of  four  problem 
types  typically  found  on  the  AP  physics  exam.  The 
source  consisted  of  20  problems.  Using  these  prob¬ 
lems,  ETS  created  sets  of  problems  to  exemplify  six 
distinct  transfer  distances  representing  systematic 
differences  between  source  and  target  problems: 
parameterization  (changing  the  parameter  values, 
but  not  the  qualitative  outcome),  extrapolation 
(changing  the  parameters  such  that  the  qualitative 
outcome  changes  as  well),  restructuring  (asking  for 
a  different  parameter),  extending  (including  dis¬ 
tracting  information),  restyling  (changing  the 
types  of  everyday  objects  involved),  and  compos¬ 
ing  (requiring  concepts  from  multiple  source  prob¬ 
lems). 

When  learners  study  for  the  AP  physics  exam, 
one  important  way  in  which  they  learn  is  by  solv¬ 
ing  problem  sets.  For  feedback,  they  often  get 
worked  solutions  —  step-by-step  explanations  typ¬ 
ically  found  in  the  back  of  textbooks.  AMF  per¬ 
forms  CBR  by  employing  worked  solutions  as  cas¬ 
es.  When  presented  with  a  new  problem,  AMF  uses 
the  many  are  called  but  few  are  chosen  structure¬ 
mapping  engine  (MAC/FAC)  (Forbus,  Gentner,  and 
Law  1995)  to  retrieve  an  analogous  worked  solu¬ 
tion  and  SME  (Falkenhainer,  Forbus,  and  Gentner 
1989)  to  reuse  its  modeling  decisions  to  construct 
a  scenario  model  of  the  new  problem.  MAC/FAC 
selects  an  analogous  case  from  the  case  base  in  two 
stages:  (1)  a  nonstructural  match  using  feature  vec¬ 
tors  whose  weights  are  proportional  to  the  number 
of  occurrences  of  each  predicate  in  a  representa¬ 
tion  and  (2)  a  structural  alignment  to  determine  its 
relational  similarity.  In  the  first  stage,  the  problem 
is  compared  to  each  case  in  the  case  base  with  the 
three  most  similar  selected  for  structural  discrimi¬ 
nation.  To  determine  the  closest  structural  match, 
SME  computes  match  scores  by  creating  an  ana¬ 


logical  mapping  between  each  selected  case  and 
the  problem,  and  the  closest  match  is  selected. 
Analogical  mappings  consist  of  correspondences 
between  the  entities  and  expressions  of  the  worked 
solution  and  problem.  From  these  correspondenc¬ 
es,  SME  creates  a  set  of  candidate  inferences,  which 
are  conjectures  about  the  problem  using  expres¬ 
sions  from  the  base,  which,  while  unmapped  in 
their  entirety,  have  subcomponents  that  are 
included  in  the  correspondences. 

Analogical  model  formulation  is  implemented 
in  the  Companion  cognitive  architecture  (Forbus, 
Klenk,  and  Hinrichs  2009),  which  is  exploring  the 
hypothesis  that  analogical  processing  is  central  to 
human  reasoning  and  learning.  After  attempting  a 
problem,  the  Companion  is  provided  with  its 
worked  solution,  which  it  retains  for  future  prob¬ 
lem  solving  by  adding  it  to  its  case  base. 

Central  to  this  approach  is  the  use  of  candidate 
inferences  generated  by  the  analogical  mapping  to 
make  modeling  decisions.  Consider  the  following 
source  problem: 

An  astronaut  on  a  planet  with  no  atmosphere 
throws  a  baseball  upward  from  near  ground  level 
with  an  initial  speed  of  4.0  meters  per  second.  If  the 
baseball  rises  to  a  maximum  height  of  5.0  meters, 
what  is  the  acceleration  due  to  gravity  on  this  plan¬ 
et?  (a)  0.8  m/s2;  (b)  1.2m/s2;  (c)  1.6m/s2;  (d)  20m/s2 

The  worked  solution  to  this  problem  includes 
seven  steps  instantiating  the  relevant  equations, 
assuming  parameter  values  and  solving  equations. 
Figure  3  includes  a  subset  of  the  representation, 
simplified  for  presentation,  of  one  step  from  the 
worked  solution  and  a  restyling  problem  in  which 
the  baseball,  astronaut,  and  planet  are  replaced  by 
a  rock,  an  alien,  and  an  asteroid,  respectively.  In 
this  worked  solution  step,  the  speed  of  the  baseball 
is  assumed  to  be  0  meters  per  second  at  the  top  of 
its  projectile  motion  event.  Analogical  model  for¬ 
mulation  uses  the  candidate  inferences  resulting 
from  this  worked  solution  step  to  make  the  mod¬ 
eling  decision  that  the  rock's  speed  at  the  top  of  its 
projectile  motion  event  is  0  meters  per  second. 
While  each  of  the  worked  solution  steps  results  in 
candidate  inferences,  analogical  model  formula¬ 
tion  applies  only  those  representing  modeling 
decisions,  such  as  instantiating  equations,  assum¬ 
ing  values,  and  checking  boundary  conditions,  to 
the  problem.  After  creating  the  scenario  model,  the 
Companion  uses  a  rule-based  problem  solver  to 
solve  for  the  sought  quantity  and  select  the  appro¬ 
priate  multiple  choice  answer. 

The  following  transfer  learning  evaluation  was 
designed  by  the  Educational  Testing  Service  and 
Cycorp  to  evaluate  a  Companion's  ability  to  trans¬ 
fer  the  knowledge  necessary  to  solve  AP  physics- 
style  problems  using  analogical  model  formula¬ 
tion.  Given  a  source  task  of  20  problems  and 
worked  solutions,  target  learning  for  six  transfer 
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(stepType  Step3  DeterminingValueFromContext) 
(stepUses  Step3  (isa  Throwingl  ThrowingAnObject)) 
(stepllses  Step3  (occursNear  Throwingl  Groundl)) 
(stepUses  Step3 
(no-GenQuantRelnFrom 

in-lmmersedFully  Planetl  Atmosphere)) 
(stepUses  Step3  (objectMovingUpwardl  BaseBalll)) 

(stepUses  Step 3  (direction  Upwardl  UpDirectly)) 
(solutionStepResult  Step3 
(valueOf 

(AtFn  ((QPQuantityFn  Speed)  BaseBalll) 

(EndFn  Upwardl)) 

(MetersPerSecond  0))) 


(groundOf  Asteroidl  Ground2) 

(performedBy  Throwing2  Alienl ) 

(no-Gen  QuantRelnFrom 

in-lmmersedFully  Asteroidl  Atmosphere) 
(eventOccursNear  Throwing2  Ground2) 
(objectThrown  Throwing2  Rockl) 
(querySentenceOf Query  Query2 
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Figure  3.  A  Portion  of  the  Representation  of  a  Source  Worked  Solution  (left)  and  Corresponding  Restyling  Problem  (right). 

(Simplified  for  readability.) 


distances  was  performed  independently  as  follows. 
The  Companion  was  given  a  series  of  five  training 
sets  each  consisting  of  a  sequence  of  four  quizzes. 
Each  quiz  consists  of  one  problem  from  each  of  the 
four  types.  After  each  quiz,  the  Companion 
received  the  worked  solutions  for  the  problems  on 
that  quiz.  After  each  training  set,  the  Companion's 
memory  was  reset.  In  the  transfer  condition,  the 
Companion  began  with  the  worked  solutions  to 
the  source  problems  and  worked  solutions  in  its 
memory.  In  the  nontransfer  condition,  the  Com¬ 
panion  began  with  zero  worked  solutions. 

The  learning  curves  grouped  by  transfer  distance 
are  shown  in  figure  4.  Averaged  across  all  the  trans¬ 
fer  distances,  the  Companion  achieved  a  95.8  per¬ 
cent  initial  advantage  due  to  the  source  problems 
and  their  worked  solutions.  On  parameterization, 
extrapolation,  restructuring,  extending,  and 
restyling  problems,  the  Companion  exhibited  per¬ 
fect  transfer.  That  is,  the  system  performed  at  ceil¬ 
ing  (100  percent)  given  just  the  source  set  worked 
solutions.  On  composing  problems,  the  system 
recorded  an  initial  advantage  of  75  percent.  All  of 
the  initial  advantages  are  statistically  significant  ( p 
<  .01). 

These  results  illustrate  that  analogical  model  for¬ 
mulation  enables  a  system  to  transfer  the  model 
formulation  knowledge  necessary  to  solve  AP 
physics-style  problems  for  the  six  transfer  distances 
shown.  The  only  failures  of  transfer  involved  limi¬ 
tations  in  the  verification  of  analogical  inferences. 
In  particular,  our  system  prevented  a  necessary 
modeling  decision  from  being  made  on  a  set  of  dif¬ 


ficult  problems  (25  percent  of  the  composing  prob¬ 
lems). 

Other  Evaluations  of  CBR 
as  Transfer  Learning  Methods 

Other  researchers  have  explored  the  evaluation  of 
CBR  as  a  transfer  learning  method.  Also  using  SME 
for  analogical  reasoning,  Hinrichs  and  Forbus 
(2011)  transfer  learned  city  management  decisions 
in  a  turn-based  strategy  game.  The  transfer  dis¬ 
tance  in  this  work  involves  changing  the  configu¬ 
ration  of  tiles  around  the  cities.  As  another  exam¬ 
ple,  CARL  (Sharma  et  al.  2007)  integrates  CBR  with 
reinforcement  learning  to  play  a  real-time  strategy 
game.  CARL  uses  reinforcement  learning  for  credit 
assignment  and  CBR  to  estimate  a  value  function 
for  a  given  task.  After  learning  in  the  source 
domain,  the  case  base  is  used  directly  in  the  target. 
CARL  was  evaluated  on  two  transfer  distances:  (1) 
the  target  scenario  swapped  the  starting  locations 
of  friendly  and  opposition  forces,  and  (2)  the  num¬ 
ber  of  scenario  entities  was  at  least  50  percent 
greater  in  the  target  scenario  than  the  source. 
Using  a  weighted  combination  of  state  features  as 
a  performance  metric,  CARL  demonstrated  a  sig¬ 
nificant  positive  initial  advantage  on  both  transfer 
distances  and  an  asymptotic  advantage  on  the  sec¬ 
ond  transfer  distance. 

Discussion 

As  transfer  learning  methods,  CBR  approaches 
have  predominately  been  applied  to  transfer  dis¬ 
tances  where  the  representational  vocabulary  of 
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Figure  4.  Average  Problem-Solving  Performance  Using  Analogical  Model  Formulation  to  Solve  AP  Physics-Style  Problems. 


the  problem  (that  is,  game  states  and  physics  sce¬ 
narios)  and  what  gets  reused  (that  is,  possible 
actions  and  modeling  decisions)  are  shared  across 
source  and  target.  This  permits  cases  from  either 
source  or  target  to  be  directly  applied  to  target 
problems.  Most  CBR  retrieval  systems  rely  on  fea¬ 
ture  similarity.  Consequently,  identifying  and 
retrieving  analogous  source  instances  is  very  diffi¬ 
cult  when  the  vocabulary  of  the  problems  and 
solutions  significantly  differs  between  source  and 
target.  Consequently,  performing  transfer  learning 
in  a  CBR  cycle  is  unlikely  to  work  for  such  transfer 
distances. 

One  of  the  original  motivations  for  CBR  research 
was  to  provide  greater  flexibility  than  rule-based 
expert  systems  for  solving  unanticipated  problems. 
While  measuring  this  flexibility  is  difficult,  transfer 
learning  provides  one  approach  for  empirically 
evaluating  a  CBR  system  by  categorizing  differ¬ 
ences  in  source  and  target  problems  by  transfer  dis¬ 
tance.  Transfer  learning  contributes  two  important 
metrics  for  evaluating  the  flexibility  of  CBR  sys¬ 
tems.  First,  the  initial  advantage  metric  empirical¬ 
ly  measures  the  flexibility  of  the  CBR  system's 
retrieval  and  reuse  mechanisms.  That  is,  the  trans¬ 
fer  distance  indicates  how  similar  problems  have 
to  be  in  order  to  retrieve  and  reuse  a  solution.  Sec¬ 
ond,  when  the  system  is  unable  to  solve  a  target 
problem  with  the  source  problems,  the  learning 
rate  measures  the  retrieval  mechanism's  ability  to 
avoid  source  cases  or  the  CBR  system's  ability  to 
perform  case-based  maintenance  (Leake  et  al. 
2001).  Empirically  evaluating  the  same  CBR  system 
across  a  range  of  transfer  distances,  as  in  the  AP 
physics  evaluation  above,  provides  information 
regarding  the  flexibility  of  the  CBR  system. 


CBR  for  Problem 
Learning:  Rush  Football 

CBR  systems  learn  by  revising  and  retaining  cases 
as  they  gain  experience.  Approaches  using  CBR  for 
the  problem  learning  transfer  learning  step  main¬ 
tain  a  case  base  for  a  set  of  problems  (source  or  tar¬ 
get).  Here,  we  present  an  example  of  this  approach 
for  both  source  and  target  problems  with  separate 
case  bases.  The  source  task,  intent  recognition 
(Sukthankar  2007),  is  to  identify  the  opposing 
team's  intent  (that  is,  play),  and  the  target  task  is  to 
control  a  player  in  the  Rush  2008  American  foot¬ 
ball  simulation  (Rush  2008). 4  Rush  simulates  a  sim¬ 
plified  version  of  American  football  with  only 
eight  players  on  each  team  and  whose  field  is  100 
by  63  yards.  Figure  5  displays  an  annotated  screen- 
shot  from  the  simulator.  The  offensive  team's 
objective  is  to  advance  the  ball  into  the  defensive 
team's  end  zone,  while  the  defensive  team's  objec¬ 
tive  is  to  prevent  this.  The  game  is  divided  into 
downs.  Before  each  down,  each  team  secretly 
selects  a  play,  which  is  a  set  of  instructions  for  each 
player  on  the  team.  Each  down  ends  when  the 
offensive  player  is  tackled  by  an  opposing  player  or 
a  forward  pass  is  incomplete. 

Case-Based  Q-Lambda 
with  Intent  Recognition 

Case-based  q-lambda  with  intent  recognition,  or 
CBQL-IR  (Aha,  Molineaux,  and  Sukthankar  2009), 
applies  CBR  separately  to  the  source  task,  intent 
recognition,  and  the  target  task,  controlling  the 
quarterback.  CBQL-IR  uses  CBR  for  both  the  source 
and  target  problems.  For  intent  recognition  prob- 
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Figure  5.  Screenshot  at  the  Beginning  of  a  Down. 


The  QB  is  the  quarterback  and  begins  the  play  with  ball.  The  wide  receivers  (WRs)  and  tight  end  (TE)  move  downfield 
to  receive  a  pass  from  the  QB.  The  running  back  (RB)  either  takes  a  handoff  from  the  QB  or  runs  down  field  to  receive 
a  pass. 


lems,  a  /c-nearest  neighbor,  or  k-NN  (Dasarathy 
1991),  is  used  to  infer  the  opponent's  play.  For  the 
QB  control  problems,  CBR  is  used  to  approximate 
the  standard  reinforcement  learning  Q-function. 

To  perform  the  source  task,  intent  recognition, 
CBQL-IR  uses  a  /c-nearest  neighbor  classifier  with 
one  example  from  each  of  the  eight  possible  defen¬ 
sive  plays.  Each  example  includes  a  feature  vector 
representing  the  movements  of  each  of  the  eight 
defensive  players  over  the  first  three  time  steps  of 
the  play.  As  a  result  of  this  source  task  learning, 
CBQL-IR  accurately  classifies  the  current  defensive 
play  after  three  time  steps. 

The  target  task  is  to  control  the  quarterback  (QB) 
by  selecting  actions,  shown  in  figure  6,  at  each 
time  step.  There  are  nine  actions  available  to  the 
quarterback:  passing  the  ball  to  WR1,  WR2,  TE,  or 
RB,  moving  in  one  of  four  directions,  and  remain¬ 
ing  stationary.  The  other  players  all  execute  the 
actions  described  in  the  offensive  and  defensive 
plays.  CBQL-IR  models  this  as  an  RL  task  with 
rewards  based  on  the  outcome  of  each  play.  At 
each  state,  CBQL-IR  uses  the  cases  with  the  most 
similar  states  to  estimate  the  expected  reward  for 
each  action.  States  are  represented  using  two  fea¬ 
tures:  the  predicted  opposing  play  and  the  current 


time  step.  At  the  end  of  the  down,  the  received 
reward  is  used  to  store  new  cases,  if  they  are  suffi¬ 
ciently  different  from  previous  ones,  or  update  the 
existing  similar  cases  accordingly. 

By  separating  source  and  target  tasks  in  this 
manner,  the  transfer  learning  evaluation  acts  as  an 
ablation  study,  measuring  the  effects  of  intent 
recognition  on  CBQL-IR's  ability  to  control  the  QB. 
In  the  transfer  condition,  the  predicted  defensive 
play  feature  was  determined  by  the  k-NN  classifier 
learned  from  training  on  source  problems.  In  the 
nontransfer  condition,  this  feature  was  assigned  a 
random  value  because  it  had  no  knowledge  from 
the  source  problems. 

The  results  are  shown  in  figure  7.  While  CBQL- 
IR  begins  with  identical  performance  in  the  two 
conditions,  it  learns  faster  and  achieves  a  higher 
ceiling  in  the  transfer  condition.  There  is  no  initial 
advantage  because  CBQL-IR  acts  randomly  when 
there  are  no  stored  cases,  as  is  true  for  any  rein¬ 
forcement  learner  before  training  occurs.  As 
detailed  in  Aha,  Molineaux,  and  Sukthankar 
(2009),  intent  recognition  can  significantly 
improve  the  system's  task  performance  for  this 
application.  To  summarize,  after  target  training, 
the  offense  gained  on  average  around  10  yards  in 
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Figure  6.  Subset  of  Actions  Available  to  the  QB  at  Each  Time  Step. 


Figure  7.  CBQL-IR's  Learning  Curves  for  Controlling  the  Rush  2008  QB  on  Repeated  Plays. 
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the  transfer  condition,  but  only  about  3  yards  in 
the  nontransfer  condition.  Therefore,  transfer 
learning  measures  the  effects  of  integrating  intent 
recognition  with  controlling  an  offensive  player. 

The  next  section  describes  other  systems  that 
use  CBR  solely  for  problem  learning  within  the 
transfer  learning  framework. 

Other  Applications  of  CBR 
for  Problem  Learning 

Related  research  on  instance  transfer  (Pan  and 
Yang  2010)  includes  approaches  that  integrate  aux¬ 
iliary  (source)  data  into  k-NN  classifiers  (Wu  and 
Dietterich  2004;  Shi  et  al.  2009).  These  approaches 
assume  an  abundance  of  source  instances  and  few 
target  instances.  When  the  known  target  instances 
are  insufficient  for  classification,  these  approaches 
use  similar  source  instances  for  classification. 
These  approaches  are  applications  of  CBR  to  target 
task  learning  because  they  distinguish  source 
instances  from  target  instances. 

Wu  and  Dietterich  (2004)  demonstrate  the  utili¬ 
ty  of  using  source  instances  when  the  target 
instances  are  insufficient.  They  minimized  the 
weighted  sum  for  two  loss  functions:  one  over  the 
target  instances,  and  the  other  using  only  the 
source  instances.  This  results  in  a  classifier  in 
which  target  instances  are  weighted  higher  than 
source  instances  for  k-NN  classification.  By  includ¬ 
ing  the  source  instances,  they  significantly 
increased  accuracies  for  classifying  leaf  species. 
While  Wu  and  Dietterich's  approach  weights  all 
source  cases  equally,  COITL  (Shi  et  al.  2009)  assigns 
weights  to  each  source  instance  based  on  their  sim¬ 
ilarity  to  the  target  task.  The  process  of  assigning 
weights  to  source  instances  is  also  an  application  of 
CBR  to  transfer  knowledge,  and  the  specifics  are 
discussed  later.  COITL  demonstrates  substantial 
improvements  in  generalization  and  robustness 
over  previous  approaches  on  a  number  of  standard 
datasets.  In  contrast  to  using  CBR  as  a  transfer 
learning  method,  each  of  these  systems  maintains 
a  distinction  between  source  and  target  instances. 

Discussion 

CBQL-IR  illustrates  how  case-based  reasoning  can 
be  used  in  different  source  and  target  tasks.  Classi¬ 
fication  of  the  opponent's  play  enabled  faster 
learning  and  higher  asymptotic  performance  when 
controlling  the  Rush  2008  quarterback's  actions. 
While  the  source  learning,  play  classification, 
informs  the  problem  representation  in  the  target 
task,  player  control,  the  case-based  reasoning  in 
the  target  task  only  reuses  solutions  from  target 
cases.  This  is  due  to  the  differences  between  source 
and  target  case  representations  and  solutions.  One 
cannot  adapt  a  play  classification  directly  into  the 
selection  of  an  action. 

When  the  source  and  target  tasks  are  drastically 


different,  as  in  CBQL-IR,  then  measuring  the  trans¬ 
fer  distance  is  not  applicable.  Instead,  transfer  learn¬ 
ing  functions  as  an  ablation  experiment  evaluating 
the  contribution  of  an  individual  component  in  an 
integrated  systems  research.  However,  when  the 
source  and  target  tasks  are  the  same,  as  in  the 
instance  transfer  approaches,  the  transfer  distance 
is  noteworthy.  In  such  approaches,  the  features  used 
for  classification  may  differ  among  source  and  target 
problems,  but  they  need  to  be  derivable  from  each 
target  instance.  Furthermore,  the  labels  must  be 
shared  among  the  source  and  target. 

CBR  for  Transferring  Knowledge: 
Linear  and  Rotational  Kinematics 

Approaches  using  CBR  to  transfer  knowledge  focus 
on  identifying  how  the  source  and  target  relate  to 
each  other.  To  illustrate  this  approach,  we  present 
an  application  of  domain  transfer  via  analogy  to 
learn  a  mapping  between  linear  and  rotational 
mechanics  (Klenk  and  Forbus  2009b).  In  this  work, 
an  analogical  mapping  is  created  between  source 
and  target,  which  supports  the  transfer  of  abstract 
source  knowledge.  As  in  the  AP  physics  work,  the 
source  and  target  consists  of  physics  problems  and 
worked  solutions.  Following  the  same  conventions 
as  the  work  in  the  CBR  as  a  Transfer  Learning 
Method  section,  the  problems  and  worked  solu¬ 
tions  are  represented  in  predicate  calculus  using 
the  ResearchCyc  ontology.  Unlike  the  AP  physics 
work,  the  system  uses  a  domain  theory  of  kine¬ 
matics  equations  defined  using  equation  schemas, 
which  specify  the  equation's  types,  quantities,  and 
applicability  conditions.  In  both  the  source  and 
target  tasks,  the  system  uses  the  available  equation 
schemas  and  rule-based  problem  solving  to  con¬ 
struct  the  scenario  model  and  solve  for  the  sought 
quantity. 

Domain  Transfer  Via  Analogy 

Domain  transfer  via  analogy  (figure  8)  learns  the 
equation  schemas  necessary  to  solve  target  prob¬ 
lems  by  cross-domain  analogy  with  a  source 
domain.  After  failing  to  solve  a  target  problem, 
domain  transfer  via  analogy  is  provided  with  its 
worked  solution.  The  inputs  to  domain  transfer  via 
analogy  are  this  worked  solution,  the  source  prob¬ 
lems'  worked  solutions,  and  the  source  equation 
schemas.  In  step  1,  domain  transfer  via  analogy 
uses  MAC/FAC  with  the  target  worked  solution  to 
retrieve  an  analogous  source  worked  solution,  and 
SME  to  create  a  set  of  correspondences  between  the 
types,  quantities,  and  relations  of  the  source  and 
target,  called  a  domain  mapping.  During  this 
process,  SME  aligns  nonidentical  relations  using 
minimal  ascension  (Falkenhainer  1988),  as 
described  in  Hinrichs  and  Forbus  (2011). 

In  step  2,  this  domain  mapping  is  used  to  ini- 
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Figure  8.  Conceptual  Model  for  Domain  Transfer  Via  Analogy. 


tialize  the  target  domain  theory.  For  each  source 
equation  schema  in  the  domain  mapping,  domain 
transfer  via  analogy  replaces  its  types,  predicates, 
and  quantities  with  the  corresponding  expressions 
from  the  domain  mapping.  For  example,  the  linear 
kinematics  (source)  domain  includes  the  equation 
d  =  v.t  +  .5 at2,  which  has  an  applicability  condition 
stating  that  the  predicate  objectTranslating  must 
hold  between  the  object  and  the  movement  event 
of  the  equation  schema.  Consider  a  domain  map¬ 
ping  resulting  from  step  1  that  includes  a  corre¬ 
spondence  between  the  predicates  objectTranslat¬ 
ing  and  objectRotating.  Therefore,  the  initialized 
equation  schema's  applicability  condition  is 
defined  in  terms  of  the  objectRotating  predicate. 
After  all  the  substitutions,  the  rotational  kinemat¬ 
ics  (target)  domain  includes  an  equation  schema 
defining  0  =  ®.t  +  .Sat2.  In  step  3,  the  target  domain 
theory  is  extended  through  an  analogy  between 
the  domain  theories  themselves.  This  permits 
domain  transfer  via  analogy  to  transfer  equation 
schemas  not  mentioned  in  the  target  worked  solu¬ 
tions.  Finally,  because  analogical  learning  is  not 
sound,  domain  transfer  via  analogy  verifies  the 
transferred  equation  schemas  by  attempting  to 


solve  the  target  problem.  If  successful,  the  target 
equation  schemas  are  assumed  to  be  correct  and 
retained  for  use  in  solving  future  target  problems. 
Otherwise,  the  transferred  knowledge  and  domain 
mapping  are  discarded. 

The  following  transfer  learning  evaluation  was 
performed  to  measure  the  effectiveness  of  domain 
transfer  via  analogy  for  transferring  knowledge 
between  linear  and  rotational  kinematics.  Each 
domain  includes  five  problems  and  worked  solu¬ 
tions.  Target  learning  was  measured  by  running 
120  trials  representing  every  possible  ordering  of 
the  test  materials.  In  the  transfer  condition,  the 
domain  transfer  via  analogy  was  provided  with  the 
five  worked  solutions  and  the  equation  schemas 
from  the  source  domain.  In  the  transfer  condition, 
after  the  system  failed  to  answer  a  target  problem 
correctly,  domain  transfer  via  analogy  was  invoked 
with  its  worked  solution.  In  the  nontransfer  con¬ 
dition,  after  failing  to  solve  a  target  problem,  the 
equation  schemas  necessary  to  solve  that  problem 
were  added  directly  to  the  system's  target  domain 
theory.  Two  experiments  were  performed:  one 
with  linear  kinematics  as  the  source  and  rotation¬ 
al  kinematics  as  the  target,  and  vice  versa. 
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The  results,  shown  in  figures  9  and  10,  graph  the 
percentage  of  problems  the  system  answered  cor¬ 
rectly  against  a  problem's  position  within  the  trial. 
In  learning  rotational  mechanics,  while  domain 
transfer  via  analogy  transferred  the  equation 
schemas  necessary  to  answer  correctly  the  remain¬ 
ing  questions  after  only  one  problem,  the  non¬ 
transfer  condition  only  answered  the  last  problem 
on  the  quiz  correctly  80  percent  of  the  time.  In 
learning  linear  kinematics,  after  any  three  prob¬ 
lems  both  the  nontransfer  and  transfer  conditions 
were  able  to  correctly  answer  the  rest  of  the  prob¬ 
lems.  An  analysis  of  the  few  linear  kinematics 
transfer  failures  indicates  that  increasing  the  num¬ 
ber  of  subevents  and  time  intervals  in  the  problem 
representation  increases  the  difficulty  in  generat¬ 
ing  a  useful  domain  mapping.  The  results  from 
both  experiments  demonstrate  that  domain  trans¬ 
fer  via  analogy  outperformed  the  nontransfer  con¬ 
dition  by  learning  faster  and  achieved  the  same  or 
higher  ceiling.  This  work  has  been  expanded  to 
demonstrate  transfer  between  linear  mechanical, 
rotational,  electrical,  and  thermal  systems  (Klenk 
and  Forbus  2009c). 

Other  Applications  of  CBR 
for  Transferring  Knowledge 

We  distinguish  these  approaches  according  to 
whether  they  perform  analogical  or  instance  trans¬ 
fer.  Analogical  transfer  approaches,  including 
domain  transfer  via  analogy,  create  mappings 
between  the  source  and  target  supporting  the 
transfer  of  abstract  source  knowledge.  Instance 
transfer  approaches  identify  source  instances  that 
are  directly  applicable  to  the  target  task. 

Analogical  Transfer  Methods.  As  discussed  earlier, 
structure  mapping  theory  (Gentner  1983)  defines 
analogy  as  an  alignment  process  between  two 
structured  representations.  The  resulting  align¬ 
ment  is  a  mapping  between  entities  and  expres¬ 
sions  of  the  two  representations,  which  satisfy  the 
analogical  process  model's  structural,  semantic, 
and  pragmatic  constraints  (Holyoak  and  Thagard 
1989).  Different  process  models  (SME,  Falkenhain- 
er,  Forbus,  and  Gentner  [1989];  LISA,  Hummel  and 
Holyoak  [2003];  AMBR,  Kokinov  and  Petrov 
[2001])  employ  slightly  different  versions  of  these 
constraints.  Analogy  research  seeks  to  understand 
the  principles  underlying  the  retrieval  of  analo¬ 
gous  cases,  the  mapping  between  them,  the  infer¬ 
ences  suggested,  and  the  integration  of  analogical 
reasoning  in  cognition.  This  alignment  process  is 
essential  for  retrieval  and  reuse  in  CBR.  Therefore, 
we  consider  the  transfer  learning  methods  below 
as  using  partial  CBR  cycles  for  transfer.  These 
approaches  differ  with  respect  to  the  model  of 
analogy,  representations,  and  transferred  knowl¬ 
edge,  which  we  summarize  in  table  2. 

Lui  and  Stone  (2006)  present  a  method  for  ana- 


Rotational  Kinematics 


Problem  Number 


Figure  9.  Rotational  Kinematics  as  Target 


Linear  Kinematics 


Figure  1 0.  Learning  Linear  Kinematics  as  Target 


logical  transfer  using  a  version  of  SME  that  was 
optimized  to  map  sets  of  qualitative  dynamic 
Bayes  nets  (QDBNs).  QDBNs  are  provided  for  the 
source  and  target  RL  problems  describing  the 
effects  of  an  action.  The  resulting  mapping  is  used 
to  transfer  Q-values  between  reinforcement  learn¬ 
ing  problems.  Transfer  over  QDBNs  improved  per¬ 
formance  in  learning  policies  for  keepaway  soccer 
games  of  various  sizes. 

Isomorphism  between  two  graph  representa¬ 
tions  underlies  most  analogy  research.  Kuhlmann 


SPRING  2011  65 


Articles 


Model  of  Analogy 

Base  and  Target  Representations 

Transferred  Knowledge 

Lui  and  Stone  (2006) 

SME-QDBN 

QDBNs  of  Task  Models 

RL  Value  Function 

Kuhlmann  and  Stone  (2007) 

Graph  Isomorphism 

Canonical  GGP  Game  Rules 

RL  Value  Function 

Klenk  and  Forbus  (2009) 

SME 

Physics  Worked  Solutions 

Equation  Schemas 

Konik  et  al.  (2009) 

GAMA 

Analyzed  GGP  Game  Rules 

Hierarchical  Skills 

Hinrichs  and  Forbus  (2011) 

SME 

Analyzed  GGP  Game  Rules 

HTN-Methods 

Table  2.  Analogical  Transfer  Methods  for  Transfer  Learning. 


and  Stone  (2007)  transform  general  game  playing, 
or  GGP  (Genesereth,  Love,  and  Pell  2005),  game 
rules  into  a  conical  form  stored  as  a  graph  for  each 
game.  Next,  they  find  graph  isomorphisms 
between  games  to  reuse  Q-value  functions  learned 
from  playing  the  source  game.  Transformations 
between  source  and  target  include  different  board 
sizes  (3x3  minichess  to  4x4  minichess)  or  inverted 
goals  (winning  tac-tac-toe  to  losing  tic-tac-toe). 

Hinrichs  and  Forbus  (2011)  use  the  static  analy¬ 
sis  of  GGP  game  rules  to  classify  game  predicates 
into  broad  classes  including  types,  quantities,  spa¬ 
tial  coordinates,  and  ordinal  relations.  This  infor¬ 
mation  enables  the  alignment  of  nonidentical 
predicates  through  minimal  ascension  and 
metamappings.  Using  the  Companion  cognitive 
architecture,  they  transfer  hierarchical  task  net¬ 
work  methods  across  five  different  transfer  dis¬ 
tances,  enabling  faster  learning  in  the  target  prob¬ 
lems. 

The  Icarus  cognitive  architecture  employs  goal- 
driven  analogical  mapping  (GAMA)  to  translate 
acquired  skills  from  source  problems  to  related  tar¬ 
get  problems  (Konik  et  al.  2009).  Icarus  explains  a 
provided  trace  from  a  source  problem  using  con¬ 
cepts  from  its  source  domain  theory.  GAMA  uses 
this  trace  pragmatically  to  constrain  the  mapping 
between  source  and  target  domain  theories  defined 
by  GGP  rules.  This  mapping  enables  the  transfer  of 
hierarchical  skills  learned  in  the  source  domain, 
thereby  accelerating  search  in  the  target  domain. 

Transfer  in  each  GGP  transfer  problem  requires 
mapping  nonidentical  predicates.  Each  of  these 
approaches  includes  higher-order  relations  to  con¬ 
strain  the  matching  process.  While  the  tier-identi- 
cality  constraint  of  structure  mapping  theory 
(Gentner  1983)  for  analogical  mapping  states  a 
strong  preference  for  identical  predicates,  it  also 
permits  nonidentical  predicates  to  match,  such  as 
those  used  in  the  works  described  in  this  section. 
Future  work  on  analogy  from  both  AI  and  psycho¬ 
logical  perspectives  will  investigate  when  and  how 
these  nonidentical  matches  may  arise. 

Instance  Transfer  Methods.  An  alternative 
approach  is  to  use  CBR  to  learn  weights  for  source 


instances  for  use  in  solving  target  problems.  For 
example,  COITL  (Shi  et  al.  2009)  assigns  weights  to 
source  instances  based  on  their  similarity  to  known 
target  instances.  A  classifier  consisting  of  known  tar¬ 
get  instances  is  used  to  classify  each  source  instance. 
If  the  classification  succeeds,  then  the  source 
instance  is  given  the  weight  of  the  labeling  confi¬ 
dence  of  the  classifier.  If  the  classifier  fails,  then  the 
source  instance's  weight  is  set  to  zero.  Transfer  is 
performed  by  adding  the  positive  weighted  source 
instances  to  the  target  (k-NN)  classifier. 

Discussion 

Domain  transfer  via  analogy  creates  an  analogical 
mapping  between  the  source  and  target  domains 
to  support  the  transfer  of  abstract  source  knowl¬ 
edge.  This  enables  the  system  to  learn  rotational 
kinematics  by  analogy  with  linear  kinematics,  and 
vice  versa.  Using  SME  and  minimal  ascension, 
domain  transfer  via  analogy  successfully  transfers 
the  kinematics  equation  schemas  necessary  for 
problem  solving  and  demonstrates  improved 
learning  over  the  nontransfer  condition.  The  trans¬ 
fer  distance  between  these  domains  includes  dif¬ 
ferent  relations,  quantities,  and  types. 

The  two  types  of  CBR  for  knowledge  transfer, 
analogical  and  instance  transfer,  are  applicable 
over  different  transfer  distances.  As  indicated  in 
the  previous  discussion  subsection,  instance  trans¬ 
fer  methods  are  applicable  for  transfer  distances 
where  source  and  target  feature  spaces  differ  but 
use  the  same  class  labels.  Analogical  transfer  sys¬ 
tems  are  applicable  for  domains  that  include  dif¬ 
ferent  relations  in  both  the  problem  and  solution 
representations.  In  fact,  the  GGP  representations 
for  source  and  target  included  only  a  small  number 
of  shared  relations.  Each  of  the  analogical 
approaches  employs  higher-order  relations  to  con¬ 
strain  the  matching  process. 

Conclusions 

CBR  research  is  the  study  of  how  and  under  what 
circumstances  knowledge  from  specific  cases  may 
be  applied  to  new  problems.  Therefore,  it  is  implic- 
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itly  tied  to  transfer  learning,  which  measures  the 
performance  difference  resulting  from  a  system's 
experience  in  a  source  task  on  a  related  target  task. 
We  identified  three  distinct  approaches  for  using 
case-based  reasoning  in  transfer  learning:  CBR  as  a 
transfer  learning  method,  CBR  for  problem  learn¬ 
ing,  and  CBR  for  transferring  knowledge  from 
source  to  target  domains.  Our  analysis  of  these 
approaches  illustrates  how  transfer  learning  and 
case-base  reasoning  inform  one  another. 

CBR  as  a  transfer  learning  method:  These  CBR  sys¬ 
tems  perform  transfer  learning  by  accumulating 
cases  and  applying  them  to  new  problems  without 
any  distinction  between  source  and  target  prob¬ 
lems.  Currently,  these  systems  have  only  been 
applied  to  the  transfer  distances  where  the  repre¬ 
sentation  vocabulary  for  problems  and  solutions 
are  shared  across  source  and  target.  In  addition  to 
performing  near  transfer,  the  transfer  learning 
framework  provides  these  systems  with  two  ways 
for  evaluating  their  utility.  First,  by  measuring  per¬ 
formance  as  target  cases  are  accumulated,  it  meas¬ 
ures  the  (potentially  negative)  impact  of  the  source 
cases  on  retrieval  and  reuse.  That  is,  as  the  agent 
accumulates  target  cases,  its  retrieval  mechanism 
should  ignore  the  source  cases  in  favor  of  the  more 
applicable  target  cases.  Second,  the  division  of 
source  and  target  problems  allows  the  initial 
advantage  metric  to  empirically  assess  the  CBR  sys¬ 
tem's  retrieval  and  reuse  mechanisms  for  a  partic¬ 
ular  transfer  distance. 

CBR  for  problem  learning:  These  integrated  sys¬ 
tems  distinguish  source  and  target  cases.  As  in 
CBQL-IR,  the  source  and  target  problem  types  may 
be  sequential  (and  differ).  CBQL-IR  uses  the  output 
from  the  (classification)  source  task  as  input  to  the 
(reinforcement  learning)  target  task.  Transfer  learn¬ 
ing  measures  the  ability  of  the  CBR  problem  learn¬ 
er  to  leverage  source  knowledge  while  performing 
the  target  task,  similar  to  an  ablation  study.  Alter¬ 
natively,  the  source  and  target  problem  types  may 
be  the  same.  Here,  the  source  instances  act  as  aux¬ 
iliary  knowledge  sources  that  complement  the  lim¬ 
ited  number  of  target  instances.  For  these  systems, 
transfer  learning  measures  the  effects  of  these  aux¬ 
iliary  cases. 

CBR  for  transferring  knowledge:  In  these  approach¬ 
es,  a  CBR  process  transfers  knowledge  from  the 
source  for  use  by  the  domain  learner  in  the  target. 
Analogical  transfer  methods  have  been  used  for 
transfer  distances  in  which  different  relations, 
types,  and  quantities  define  the  source  and  target 
problems  and  solutions.  These  approaches  have 
been  integrated  with  a  variety  of  domain  learners 
for  classification,  reinforcement  learning,  hierar¬ 
chical  skill  learning,  and  hierarchical  task  network 
method  learning.  Furthermore,  they  provide  a  link 
between  machine  learning  and  cognitive  science. 
We  hope  this  communication  continues,  as  an 


interdisciplinary  approach  to  studying  intelligence 
is  essential  to  scientific  progress  in  AI. 

In  the  Transfer  Learning  subsection,  we  dis¬ 
cussed  an  important  contribution  of  transfer  learn¬ 
ing:  the  categorization  of  differences  between 
source  and  target  problems  by  transfer  distance. 
We  expect  future  researchers  to  define  and  use 
transfer  distances  to  better  understand  the 
strengths  and  weaknesses  of  their  CBR  approaches. 
Pan  and  Yang's  (2010)  categorizations  are  a  strong 
step  forward  for  characterizing  transfer  learning 
approaches,  and  we  suggest  four  additional  dimen¬ 
sions. 

The  first  is  direct  applicability  of  source  knowl¬ 
edge.  Knowledge  learned  on  source  problems  is 
directly  applicable  if  it  assists  with  solving  target 
problems  with  no  additional  transformations.  For 
CBQL-IR,  the  source  and  target  problems  differ,  but 
the  knowledge  gained  in  the  source  was  from  the 
same  frame  of  reference,  and  thus  was  directly 
applicable.  On  the  other  hand,  the  analogical 
transfer  methods  transform  the  source  knowledge 
for  application  in  the  target. 

The  second  dimension  is  whether  the  learning 
problem  types  are  consistent.  For  example,  Wu  and 
Dietterich's  (2004)  learning  task  is  classification  for 
both  source  and  target  problems.  As  an  example  of 
inconsistent  types,  CBQL-IR's  (Aha,  Molineaux, 
and  Sukthankar  2009)  source  task  is  classification 
and  target  task  is  reinforcement  learning.  This  can 
impact  ease  of  transferring  learned  source  knowl¬ 
edge. 

The  third  dimension  concerns  problem  labeling. 
Transfer  learning  approaches  typically  assume  that 
target  problems  will  be  identified  distinctly  from 
source  problems.  However,  some  research  in  CBR 
as  a  transfer  learning  method  relaxes  the  assump¬ 
tion  that  these  different  kinds  of  problems  are  dis¬ 
tinctly  identified;  all  new  problems  are  presented 
as  the  same,  regardless  of  whether  they  are  source 
or  target  problems. 

The  fourth  dimension  concerns  multiple  repre¬ 
sentations.  In  complex  domains,  such  as  strategy 
simulations,  AI  systems  must  integrate  representa¬ 
tions  for  planning,  execution  monitoring,  and  spa¬ 
tial  reasoning.  The  incorporation  of  multiple  rep¬ 
resentations,  such  as  hierarchical  task  network 
methods  and  spatial  diagrams,  may  assist  transfer 
learning  approaches  in  finding  and  exploiting  sim¬ 
ilarities  between  the  domains  (Klenk  2009).  While 
transfer  learning  has  not  explicitly  explored  this 
dimension,  Klenk  et  al.  (2005)  applies  analogical 
model  formulation  to  solving  physical  reasoning 
problems  involving  spatial  and  conceptual  repre¬ 
sentations. 

In  summary,  research  on  CBR  has  varied  among 
these  dimensions.  Future  work  remains  on  meas¬ 
uring  how  these  dimensions  characterize  transfer 
distance,  and  how  transfer  distance  can  be  used  to 
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predict  a  given  CBR  method's  transfer  learning  per¬ 
formance.  Transfer  learning  is  not  an  end  of  itself; 
it  is  an  important  stepping  stone  to  developing 
agents  that  exist  over  a  long  period  of  time  and 
learn  to  solve  a  wide  variety  of  problems.  While 
transfer  learning  methods  enable  the  reuse  of 
knowledge  learned  in  the  source  to  improve  future 
performance,  in  autonomous  situations,  there  is  an 
additional  problem  of  selecting  an  analogous 
source.  This  problem  has  received  infrequent 
attention  in  the  transfer  learning  literature. 

In  addition  to  purely  autonomous  agents,  trans¬ 
fer  learning  is  an  important  technology  for  collab¬ 
orative  agents  that  work  with  users  on  a  wide  range 
of  problems.  These  agents  could  leverage  transfer 
learning  methods  directly  to  exploit  user-provided 
cross-domain  analogies,  such  as  "rotational 
motion  is  analogous  to  linear  motion"  or  "a  heat 
pump  is  like  a  water  pump."  Further  dialogue 
between  agents  and  users  could  explore  the  analo¬ 
gy  between  the  domains  to  highlight  important 
correspondences  and  identify  the  causes  of  nega¬ 
tive  transfer.  While  difficult  to  construct,  advances 
in  transfer  learning  may  substantially  increase  the 
competence  of  intelligent  agents. 
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Notes 

1.  Transfer  learning  can  also  support  CBR  methods  in  a 
variety  of  ways,  but  we  leave  that  analysis  for  future  work. 

2.  For  more  information,  see  the  CBR  Wiki,  cbrwiki.fdi. 
ucm.es/wiki/index.php/Main_Page. 

3.  CBR  for  problem  learning  and  CBR  to  transfer  knowl¬ 
edge  are  not  mutually  exclusive.  An  integrated  system 
could  use  two  separate  CBR  cycles,  one  for  problem  learn¬ 
ing  and  one  to  transfer  knowledge,  to  perform  transfer 
learning. 

4.  See  www.knexusresearch.com/projects/rush. 
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